Confident Learning: Estimating Uncertainty in Dataset Labels

Learning exists in the context of data, yet notions confidence typically focus on model predictions, not label quality. Confident learning (CL) is an alternative approach which focuses instead quality by characterizing and identifying errors datasets, based principles pruning noisy counting with probabilistic thresholds to estimate noise, ranking examples train confidence. Whereas numerous studies have developed these independently, here, we combine them, building assumption a class-conditional noise process directly joint distribution between (given) labels uncorrupted (unknown) labels. This results generalized CL provably consistent experimentally performant. We present sufficient conditions where exactly finds errors, show performance exceeding seven recent competitive approaches for CIFAR dataset. Uniquely, framework coupled specific data modality or (e.g., use find several presumed error-free MNIST dataset improve sentiment classification text Amazon Reviews). also employ ImageNet quantify ontological class overlap estimating 645 missile images are mislabeled as their parent projectile), moderately increase accuracy ResNet) cleaning prior training. These replicable using open-source cleanlab release.